Convergence and Asymptotic Normality of Variational Bayesian Approximations for Expon
نویسندگان
چکیده
We study the properties of variational Bayes approximations for exponential family models with missing values. It is shown that the iterative algorithm for obtaining the variational Bayesian estimator converges locally to the true value with probability 1 as the sample size becomes indefinitely large. Moreover, the variational posterior distribution is proved to be asymptotically normal.
منابع مشابه
Convergence and asymptotic normality of variational Bayesian approximations for exponential family models with missing values
We study the properties of variational Bayes approximations for exponential family models with missing values. It is shown that the iterative algorithm for obtaining the variational Bayesian estimator converges locally to the true value with probability 1 as the sample size becomes indefinitely large. Moreover, the variational posterior distribution is proved to be asymptotically normal.
متن کاملFrequentist Consistency of Variational Bayes
A key challenge for modern Bayesian statistics is how to perform scalable inference of posterior distributions. To address this challenge, variational Bayes (vb) methods have emerged as a popular alternative to the classical Markov chain Monte Carlo (mcmc) methods. vb methods tend to be faster while achieving comparable predictive performance. However, there are few theoretical results around v...
متن کاملGaussian Approximations for Probability Measures on R
This paper concerns the approximation of probability measures on Rd with respect to the Kullback-Leibler divergence. Given an admissible target measure, we show the existence of the best approximation, with respect to this divergence, from certain sets of Gaussian measures and Gaussian mixtures. The asymptotic behavior of such best approximations is then studied in the frequently occuring small...
متن کاملOn the properties of variational approximations of Gibbs posteriors
The PAC-Bayesian approach is a powerful set of techniques to derive non-asymptotic risk bounds for random estimators. The corresponding optimal distribution of estimators, usually called the Gibbs posterior, is unfortunately often intractable. One may sample from it using Markov chain Monte Carlo, but this is usually too slow for big datasets. We consider instead variational approximations of t...
متن کاملApproximate Message Passing for Bilinear Models
Approach: We take a Bayesian approach to the inference problems (in particular, posterior estimation) that revolve around the bilinear model (1). In particular, we leverage the approximate message passing (AMP) framework of [2], [3] and extend it to the bilinear domain. Compared to Bayesian approaches that rely on Gibbs sampling methods or variational inference, the AMP framework allows us to f...
متن کامل